Biography

I am currently a master student at the Department of Computer Science and Technology, Nanjing University, and a member of BigAI Group, which is under the supervision of professor Wu-Jun Li.

In June 2022, I received my B.Sc. degree from the School of Computer Science at Wuhan University (WHU). In the same Year, I was admitted to pursue my Ph.D. degree without entrance examination.

Research Interests

1. Deep Learning
2. Distributed Learning
3. AI Infrastucture

Awards

Gold Medal of International Colligate Programming Contest (ICPC), East Asia Regional, 2021.05

Working Experiences

Deep Learning Framework R&D Intern. Ant Group Co. (2024.06 - 2024.09)

• Constructed an end-to-end pipeline of RLHF procedure (including SFT, RM, DPO) based on Megatron-LM framework for large language model developed by the NLP team, along with full unit tests to guarantee the correctness.
• Conducted a survey on the cutting-edge work of Sequence Parallelism, studied related open-source repository and preformed performance tests, finally delivered a technology report to the whole team.
• Cooperated with a multimodal model team, solved the needs in the training process of their model. Including optimizing with parallelism technology (Deepspeed-Zero, DDP), monitoring FID scores while training etc.

C++ Development Engineer Intern. Ant Group Co. (2021.06 - 2024.09)

• Develop the CUDA kernal API of the on-going project.
• Participate in the development of a recommendation system (Based on C++).

Publications

Hao Lin*, Ke Wu*, Jie Li, Jun Li, and Wu-Jun Li†: UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming. arXiv2307.16375, 2023. [PDF]

(*: equal contribution. †: corresponding author.)